ImageCLEF 2020: Multimedia Retrieval in Lifelogging, Medical, Nature, and Security Applications

This paper presents an overview of the 2020 ImageCLEF lab that will be organized as part of the Conference and Labs of the Evaluation Forum - CLEF Labs 2020 in Thessaloniki, Greece. ImageCLEF is an ongoing evaluation initiative (run since 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2020, the 18th edition of ImageCLEF will organize four main tasks: (i) a Lifelog task (videos, images and other sources) about daily activity understanding, retrieval and summarization, (ii) a Medical task that groups three previous tasks (caption analysis, tuberculosis prediction, and medical visual question answering) with new data and adapted tasks, (iii) a Coral task about segmenting and labeling collections of coral images for 3D modeling, and a new (iv) Web user interface task addressing the problems of detecting and recognizing hand drawn website UIs (User Interfaces) for generating automatic code. The strong participation, with over 235 research groups registering and 63 submitting over 359 runs for the tasks in 2019 shows an important interest in this benchmarking campaign. We expect the new tasks to attract at least as many researchers for 2020

ImageCLEF 2022: Multimedia Retrieval in Medical, Nature, Fusion, and Internet Applications

ImageCLEF is part of the Conference and Labs of the Evaluation Forum (CLEF) since 2003. CLEF 2022 will take place in Bologna, Italy. ImageCLEF is an ongoing evaluation initiative which promotes the evaluation of technologies for annotation, indexing, and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In its 20th edition, ImageCLEF will have four main tasks: (i) a Medical task addressing concept annotation, caption prediction, and tuberculosis detection; (ii) a Coral task addressing the annotation and localisation of substrates in coral reef images; (iii) an Aware task addressing the prediction of real-life consequences of online photo sharing; and (iv) a new Fusion task addressing late fusion techniques based on the expertise of the pool of classifiers. In 2021, over 100 research groups registered at ImageCLEF with 42 groups submitting more than 250 runs. These numbers show that, despite the COVID-19 pandemic, there is strong interest in the evaluation campaign

Overview of the ImageCLEF 2021: Multimedia Retrieval in Medical, Nature, Internet and Social Media Applications

This paper presents an overview of the ImageCLEF 2021 lab that was organized as part of the Conference and Labs of the Evaluation Forum – CLEF Labs 2021. ImageCLEF is an ongoing evaluation initiative (first run in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2021, the 19th edition of ImageCLEF runs four main tasks: (i) a medical task that groups three previous tasks, i.e., caption analysis, tuberculosis prediction, and medical visual question answering and question generation, (ii) a nature coral task about segmenting and labeling collections of coral reef images, (iii) an Internet task addressing the problems of identifying hand-drawn and digital user interface components, and (iv) a new social media aware task on estimating potential real-life effects of online image sharing. Despite the current pandemic situation, the benchmark campaign received a strong participation with over 38 groups submitting more than 250 runs

The 2021 ImageCLEF Benchmark: Multimedia Retrieval in Medical, Nature, Internet and Social Media Applications

This paper presents the ideas for the 2021 ImageCLEF lab that will be organized as part of the Conference and Labs of the Evaluation Forum — CLEF Labs 2021 in Bucharest, Romania. ImageCLEF is an ongoing evaluation initiative (active since 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2021, the 19th edition of ImageCLEF will organize four main tasks: (i) a Medical task addressing visual question answering, a concept annotation and a tuberculosis classification task, (ii) a Coral task addressing the annotation and localisation of substrates in coral reef images, (iii) a DrawnUI task addressing the creation of websites from either a drawing or a screenshot by detecting the different elements present on the design and a new (iv) Aware task addressing the prediction of real-life consequences of online photo sharing. The strong participation in 2020, despite the COVID pandemic, with over 115 research groups registering and 40 submitting over 295 runs for the tasks shows an important interest in this benchmarking campaign. We expect the new tasks to attract at least as many researchers for 2021

Overview of ImageCLEFtuberculosis 2022 ::CT-based cavern detection and report

ImageCLEF is a part of the Conference and Labs of the Evaluation Forum (CLEF) initiative and includes avariety of tasks dedicated to multimodal image information retrieval, including image classification and annotation. The tuberculosis (TB) task is one of the ImageCLEF tasks that started in 2017 and changed from year to year with more complex challenges. The 2022 edition was dedicated to the automatic analysis of caverns, and included two subtasks: cavern region detection and cavern reporting. In 2022, 6 groups from 5 countries submitted at least one successful run for at least one subtask. This paper describes the TB task setup, data and the approaches of the participants

Overview of ImageCLEFtuberculosis 2021 ::CT-based tuberculosis type classification

ImageCLEF is a part of the Conference and Labs of the Evaluation Forum (CLEF) initiative and includes a variety of tasks dedicated to multimodal image information retrieval, including image classification and annotation. The tuberculosis (TB) task is one of the ImageCLEF tasks which started in 2017 and changed from year to year. The 2021 edition was dedicated to the automatic classification of five TB types: Infiltrative, Focal, Tuberculoma, Miliary, Fibro-cavernous. The task itself repeated one of the original subtasks from 2017 but the dataset was significantly changed. In 2021, 11 groups from 9 countries participated in the task and submitted at least one successful run. The task results can be compared to the TB type classification task results in the 2017 and 2018 editions. Although top scores were not improved compared to the previous editions, the participants’ results allow us to analyze the effectiveness of applying recent deep learning approaches to the task

Overview of ImageCLEF tuberculosis 2020:automatic CT-based report generation

ImageCLEF is a part of the Conference and Labs of the Evaluation Forum (CLEF) initiative and presents a set of image information retrieval tasks. ImageCLEF was historically focused on the variety of multimodal image classification, retrieval and annotation tasks. The tuberculosis task started in ImageCLEF in 2017 and changed from year to year. This year’s edition was dedicated to the automatic generation of a lung-wise CT report (CTR) based on three relevant CT findings. This year 9 groups from 8 countries participated in the task and submitted results. This year’s task is similar to the CTR (CT Report) subtask from the previous year, so it is possible to compare the results almost directly. Impressive improvement of the results was obtained with 0.92 (+0.10) average Area Under ROC-curve (AUC) and 0.89 (+0.20) minimum AUC for the three CT findings proposed

Overview of the ImageCLEF 2022 : multimedia retrieval in medical, social media and nature applications

This paper presents an overview of the ImageCLEF 2022 lab that was organized as part of the Conference and Labs of the Evaluation Forum – CLEF Labs 2022. ImageCLEF is an ongoing evaluation initiative (first run in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2022, the 20th edition of ImageCLEF runs four main tasks: (i) a medical task that groups two previous tasks, i.e., caption analysis and tuberculosis prediction, (ii) a social media aware task on estimating potential real-life effects of online image sharing, (iii) a nature coral task about segmenting and labeling collections of coral reef images, and (iv) a new fusion task addressing the design of late fusion schemes for boosting the performance, with two real-world applications: image search diversification (retrieval) and prediction of visual interestingness (regression). The benchmark campaign received the participation of over 25 groups submitting more than 258 runs

Overview of the ImageCLEF 2020: Multimedia Retrieval in Medical, Lifelogging, Nature, and Internet Applications

This paper presents an overview of the ImageCLEF 2020 lab that was organized as part of the Conference and Labs of the Evaluation Forum - CLEF Labs 2020. ImageCLEF is an ongoing evaluation initiative (first run in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2020, the 18th edition of ImageCLEF runs four main tasks: (i) a medical task that groups three previous tasks, i.e., caption analysis, tuberculosis prediction, and medical visual question answering and question generation, (ii) a lifelog task (videos, images and other sources) about daily activity understanding, retrieval and summarization, (iii) a coral task about segmenting and labeling collections of coral reef images, and (iv) a new Internet task addressing the problems of identifying hand-drawn user interface components. Despite the current pandemic situation, the benchmark campaign received a strong participation with over 40 groups submitting more than 295 runs

Overview of the ImageCLEF 2020 ::multimedia retrieval in medical, lifelogging, nature, and internet applications

This paper presents an overview of the ImageCLEF 2020 lab that was organized as part of the Conference and Labs of the Evaluation Forum - CLEF Labs 2020. ImageCLEF is an ongoing evaluation initiative (_rst run in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2020, the 18th edition of ImageCLEF runs four main tasks: (i) a medical task that groups three previous tasks, i.e., caption analysis, tuberculosis prediction, and medical visual question answering and question generation, (ii) a lifelog task (videos, images and other sources) about daily activity understanding, retrieval and summarization, (iii) a coral task about segmenting and labeling collections